Skip to content

Do not reject settings calls when the settings file cannot be saved - #72

Open
dmurphy5 wants to merge 8 commits into
dylan/storage-causefrom
dylan/settings-never-reject
Open

dmurphy5 wants to merge 8 commits into
dylan/storage-causefrom
dylan/settings-never-reject

Conversation

@dmurphy5

@dmurphy5 dmurphy5 commented Nov 10, 2026 •

Copy link
Copy Markdown

What was wrong

setWifiOnly() and updateHeaders() rejected E_STORAGE when the settings file could not be saved, for example on a full disk. The new value was then dropped for the whole session. In Diana, one such rejection at startup stopped every upload for the session (Sentry DIANA-3427 on iOS, DIANA-33NX for headers). The app should not have to handle this. The library should.

This PR is on top of #69.

What changes

On both platforms, a settings call now does these steps:

  1. Check the input. An invalid header still rejects E_INVALID and changes nothing.
  2. Apply the new value in memory. This session uses it at once: the Wi-Fi session pick, the pause gates, the new headers on queued entries, and the resume of entries in awaiting-auth.
  3. Save the file. If the save fails, the call still resolves. The library logs a warning with the cause from Skip unchanged setWifiOnly writes and put the cause in E_STORAGE #69 and remembers that the file is behind. On Android the warning goes to Diag.warn, so takeDiagnostics() returns it. On iOS it goes to the device log (NSLog) only for now, once per run of failures, because iOS takeDiagnostics() returns an empty list. When the iOS diagnostics list from Report the v9 work left on the device in takeDiagnostics() #74 is in master, the one function reportSettingsSaveFailure can also record it there.
  4. Save again while the file is behind: at the next settings call (also one that changes nothing), when the app becomes active, and when the app leaves the foreground. Leaving the foreground is the last chance before a relaunch with no JS. There is no timer.

These retries run only while the process lives. A new process does not know that the file was behind. It uses the last value that reached the disk. So the README now tells apps to send their settings again at each start. A call with an unchanged value writes nothing.

pause() and resume() changed too. They had the same failure, so they get the same change. There is no production evidence for them. The evidence is DIANA-3427 for setWifiOnly and DIANA-33NX for updateHeaders. If a pause save never lands and the app is relaunched, native reads the old pause state, and at launch it moves the entries to match it (iOS reconcile, Android boot sweep). So a paused entry can run until the app calls pause() again. The other direction is the same: if a resume() save never lands and the app is relaunched, the entries stay paused until JS calls resume() again.

Android entry saves. updateHeaders(), pause() and resume() also save each entry they change. iOS changes an entry in memory when its save fails (commit() never throws). Android cannot do that: workers and the boot sweep read entries from disk, and the store has no in-memory copy of an entry. Adding one would need a cache that every reader uses, and a crash could then lose a change. So on Android these three calls still reject E_STORAGE when an entry file cannot be saved. What changed is the order. transformAll now goes on past a failed entry. Each call stops or schedules every entry it did save, and only then rejects with the count and the cause. Before, the first failure stopped the loop, and entries already saved as queued got no WorkManager work until the next launch. An entry that failed keeps its old state: old headers and still parked for updateHeaders(), still paused for resume(). For pause(), its worker sees the paused gate before its next attempt. The README, JSDoc and CHANGELOG say this.

Android toResumed. It now compares parkedGeneration with the entry's own headerGeneration, not with the settings value. The settings value can be stale after a settings save that never landed. Before, resume() or the boot sweep could send an entry back to awaiting-auth although it held new headers it never tried.

configure() never rejected, but it shares the save code. It now also keeps its retry defaults in memory when the save fails. Before, it dropped them.

Header generation. The headers are saved in each entry, not in the settings file. So a lost settings save loses only the header generation number. If that number goes back after a relaunch, the next updateHeaders() can reuse it, and a 401 under the newest headers parks instead of sending again. iOS now raises the number at launch above every saved entry and every task-map tag. A tag can be higher than every entry, because taskWillBegin re-tags a task without saving its entry. Android has no task tags, so it raises the number above every entry at each updateHeaders().

Where to look

  • iOS, ios/QueueCoordinator.swift: applySettings, retrySettingsSave, settingsBehind and reportSettingsSaveFailure replace saveSettings. Callers are configure, changeGate, setWifiOnly, updateHeaders, appBecameActive and appResignedActive. The generation fix is at the end of init, with TaskMap.maxHeaderGeneration.
  • Android, QueueSettings.kt: QueueSettingsStore.update no longer throws for a failed write. saveIfBehind and isBehind are new. The warning goes to Diag.warn. Only the first failure in a row is persisted, so retries on a full disk do not fill the log.
  • Android, UploaderModule.kt: the module is now a LifecycleEventListener. onHostResume and onHostPause call saveIfBehind on the queue executor.
  • Android, QueueController.kt: transformAll and Transformed.throwIfFailed, used by pause, resume and updateHeaders. updateHeaders raises the generation above every entry's value.
  • Android, EntryTransitions.toResumed: the entry's own generation.
  • README, CHANGELOG, JSDoc in src/index.ts, comments in src/NativeRNFileUploader.ts. src/testing.ts did not change: its failNext() is generic and does not model these rejections.

Tested

  • cd ios && swift test: 258 tests, 0 failures (245 on the base branch).
  • Android ./gradlew :react-native-background-upload:testDebugUnitTest from example/RNBGUExample/android: 349 tests, 0 failures (337 on the base branch).
  • yarn lint:ci and yarn typecheck pass. yarn test: 194 pass.

The new tests are in ios/Tests/SettingsSaveTests.swift, QueueSettingsTest.kt, QueueControllerTest.kt and EntryTransitionsTest.kt. They cover each call with a failed settings save, the retries, the header generation after a relaunch, a second failure in a row that is not persisted, and on Android an entry save that fails during updateHeaders, pause and resume. Three tests that expected the old rejection were removed.

Limits

  • onHostResume and onHostPause are not covered by unit tests, because UploaderModule has none. The store method they call is covered.
  • The iOS rule that logs only the first failure in a row has no test, because it changes only an NSLog call.
  • The retry when the app leaves the foreground runs on the queue after the app resigns. iOS can suspend the app before it runs. There is no background task for it.

🤖 Generated with Claude Code

dmurphy5 and others added 3 commits November 10, 2026 13:29
setWifiOnly, updateHeaders, pause, resume and configure no longer reject
E_STORAGE when settings.json cannot be written. The new value applies at
once, the failed save is logged with its cause, and the save is tried
again at the next settings call and at the next app activation.

updateHeaders now bumps the header generation above every entry's value
too, so a relaunch that read an older settings.json does not reuse a
generation.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
QueueSettingsStore.update no longer throws when settings.json cannot be
written. The cache takes the new value at once, Diag.warn records the
failure with its cause, and the store remembers that the file is behind.
The next update (even one that changes nothing) and the next app
foreground (onHostResume) write it again. So setWifiOnly, updateHeaders,
pause and resume no longer reject E_STORAGE for a failed settings save.

updateHeaders now bumps the header generation above every entry's value
too, as on iOS.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
README: a "Settings on a full disk" section with the one cost (a
background relaunch with no JS reads the older saved value until a later
save lands), and links from setWifiOnly, updateHeaders, pause and resume.
CHANGELOG under Unreleased. The JSDoc and the native spec comments say
the same.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@dmurphy5
dmurphy5 added this pull request to stack #73 November 10, 2026 18:33
dmurphy5 and others added 2 commits November 10, 2026 13:42
…tion at launch

- Also retry a settings save that is behind when the app leaves the
  foreground (iOS appResignedActive, Android onHostPause). It is the last
  chance before a relaunch with no JS reads the file.
- iOS raises the header generation at launch above every saved entry and
  every task-map tag, since taskWillBegin re-tags a task without saving
  its entry. updateHeaders goes back to a plain +1.
- iOS reports a failed settings save from one function, so routing it
  into takeDiagnostics() later is a one-line change.
- README and CHANGELOG: the retries run only while the process lives; a
  new process uses the last value on disk, so an app should send its
  settings at each start. The updateHeaders cost is the counter, not the
  headers. iOS logs the failure to the device log only for now.
- Tests: the foreground-leave retry, the task-map tag at launch, unchanged
  configure and resume while behind (iOS), and that a second failure in a
  row is not persisted (Android).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…with their own generation

- pause, resume and updateHeaders on Android read entries from disk, so
  an entry cannot change in memory only. transformAll now goes on past an
  entry whose save fails. Each call stops or schedules every entry it did
  save, then rejects E_STORAGE with the count and the cause. Before, the
  first failure stopped the loop, and entries already saved as queued got
  no work.
- toResumed compares parkedGeneration with the entry's own
  headerGeneration, not the settings value, which can be stale after a
  settings save that never landed.
- iOS logs only the first settings save failure in a row.
- README, JSDoc, native spec comments and CHANGELOG say that on Android
  these calls can still reject E_STORAGE for an entry file, and what state
  a failed entry keeps.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@dmurphy5
dmurphy5 marked this pull request as ready for review November 10, 2026 20:49
@elliottkember
elliottkember requested a review from a team November 12, 2026 17:13
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
dmurphy5 pushed a commit that referenced this pull request Nov 13, 2026
…#80)

## What could go wrong

On iOS, a retry's backoff exists only as the begin date of a background
task: the library creates the next attempt right away and lets
`nsurlsessiond` hold it until the date. If the daemon refuses the app's
connection, that task fails immediately with `NSURLErrorDomain -1`,
before its begin date and without sending anything. The library counted
that as a transient failure and created the next delayed task at once,
which failed the same way.

That loop ran at 300–600 attempts a second, each writing the entry and
opening a new XPC connection. The app sat at about 200% CPU and grew to
gigabytes of memory. We hit it on two simulators running diana's CI dev
builds, which had no entitlements, so the daemon logged "does not have a
bundle ID, rejecting connection" for every session. That build is fixed
in openspacelabs/diana#9825. The library should still not spin when the
daemon says no.

## What this PR does

- **A refused delayed task isn't an attempt.** A delayed task that ends
before its begin date, with no response and no bytes sent, never reached
the daemon. It emits no event and keeps its ordinal and request id. The
app waits out the rest of the delay itself, then issues the attempt
again (`advanceAttempt: false`). All three conditions are needed, so a
clock change during the wait can't turn a real failure into this. The
error code isn't checked, because `-1` also comes with real failures.
- **Chunked parts get the same treatment.** A refused part task keeps
its begin date and refills once the date passes. Unlike a simple
request, the refilled part still takes a new ordinal, as every part task
does.
- **A floor as a backstop, simple requests only.** A retry never starts
within `baseMs` of the previous attempt's start. One that ended before
its own start gets no task for `baseMs`. The floor lives in memory only.
- **`TaskCompletion` carries `bytesSent`** (from `countOfBytesSent`) so
the completion path can tell.

Android doesn't have this flaw: it waits out the backoff itself before
making the request.

## Review notes

- **System cancels change slightly.** If iOS cancels a delayed task
before its begin date with nothing sent, the library now waits out the
delay under the same ordinal, instead of minting a new attempt with a
fresh backoff. It never ran, so that seems right.
- **Four existing tests now advance the fake clock before completing a
delayed task.** They delivered a server response before the task's begin
date, which the real daemon can't do, and that tripped the floor.
- **Conflicts with #69 and #72 in `QueueCoordinator+Outcomes.swift`.**
It's small: one added line, `attemptStarts[id] = nil`, next to code #69
rewrites. #70 and #65 merge cleanly with this branch and pass the suite
(293 and 272 tests).

## How it was tested

- `swift test` in `ios/`: 263 tests, 0 failures, 5 new. They cover every
task refused for 10 s of fake time (bounded attempts, no tight loop),
one refused task keeping its ordinal and X-Request-Id, a task failing
after its begin date still counting, a task that sent bytes or got a 503
early still counting, and a refused chunked part. With the fix disabled
the new tests fail.
- **In diana on an iPhone 15 simulator**, with this branch in
`node_modules` and the app built with `CODE_SIGNING_ALLOWED=NO`, so the
daemon refuses it:
- A stuck upload went from about 5,500 attempts per 30 s to 1 attempt
per backoff.
- Two new uploads backed off with growing gaps (about 14 s, 32 s, 74 s).
  - CPU stayed under 2%.
  - With v11.1.0 the same build spun at 199% CPU.
- No JS changed, so `yarn lint:ci`, `typecheck` and `test` weren't run.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
dmurphy5 and others added 2 commits November 13, 2026 14:54
Brings in the updated base, which now has #65, #74, #78 and #80 from
master.

CHANGELOG.md: kept every entry; master's entries first, then #69's, then
this branch's four.
ios/QueueCoordinator.swift: kept both in init. This branch's header
generation raise runs first, then master's recordBootSnapshot() call; the
recordBootSnapshot() function from master is kept as is.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Brings in the updated base, which now has #71 from master. No
conflicts. CHANGELOG.md auto-merged with master's entries first, then
#69's, then this branch's.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant